fix(testing): top-level hook outside describe no longer inflates step count#7138
fix(testing): top-level hook outside describe no longer inflates step count#7138fibibot wants to merge 1 commit into
describe no longer inflates step count#7138Conversation
…ep count A top-level `beforeAll`/`afterAll`/`beforeEach`/`afterEach` call creates a synthetic "global" `TestSuite` to host the hook. Previously that suite was also eagerly registered as a top-level `Deno.test`, so any nested `describe` would be reported as a child step of `global` — inflating the step count by one and changing the test output. Now the synthetic suite is only registered with `Deno.test` if a top-level `it()` is actually added to it. Otherwise, each child `describe` is promoted to its own `Deno.test` and inherits the global hooks at run time (so `beforeAll`/`afterAll` wrap the nested tests, and the `active` stack picks up the global `beforeEach`/`afterEach`). Fixes #7056
|
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #7138 +/- ##
=======================================
Coverage 94.61% 94.61%
=======================================
Files 634 634
Lines 51822 51866 +44
Branches 9336 9348 +12
=======================================
+ Hits 49029 49072 +43
Misses 2218 2218
- Partials 575 576 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
bartlomieju
left a comment
There was a problem hiding this comment.
Nice fix — the core mechanic (lazy synthetic + promotion with a syntheticParent back-reference) is sound, and the ordering test is a good lock-in. Left inline comments on a few spots. The two things I'd push on before merging are: (1) a regression test for the mixed case (top-level hook + top-level it() + nested describe) that the PR body documents, and (2) a quick check on it.only interaction with a promoted describe, since only propagation now skips the synthetic via describe.suite. The rest are nits.
| */ | ||
| protected isSynthetic: boolean; | ||
| /** | ||
| * For a child of a synthetic global suite, this points back to the synthetic | ||
| * suite so its hooks can be invoked around the child's tests at run time. | ||
| */ | ||
| protected syntheticParent: TestSuiteInternal<T> | null; | ||
| #registeredOptions: Deno.TestDefinition | undefined; |
There was a problem hiding this comment.
Two small things on these fields:
- Both are written only in the constructor — please mark them
readonly(or use a#private prefix to match#registeredOptionsjust below). It signals immutability and prevents accidental mutation. syntheticParentreads slightly off because the parent is what's synthetic, not the field itself. Something likeparentSyntheticorglobalHookSuitewould be clearer. Taste call.
Also worth noting in the syntheticParent JSDoc that this is an asymmetric linkage — normal parents are reached via describe.suite, only synthetic globals use this back-reference. Future readers will wonder why two pointers exist.
| if (testSuite && testSuite.isSynthetic) { | ||
| // Promote: a child describe of the synthetic global is registered as its | ||
| // own top-level Deno.test rather than as a step of the global suite. The | ||
| // child inherits the global's hooks at run time via syntheticParent. | ||
| this.syntheticParent = testSuite; | ||
| this.registerAsDenoTest(); |
There was a problem hiding this comment.
it.only propagation worth double-checking here. addingOnlyStep walks parents via suite.describe.suite, which is not set for a promoted child (its synthetic linkage lives on syntheticParent). Two cases:
it.only()inside the promoted describe → works, becausehasOnlyStepis set duringfn()beforeregisterAsDenoTest()reads it.- Top-level
it.only()plus a separatedescribe(...)→ the synthetic becomes a realDeno.testmarkedonly, but the promoted describe is neitheronlynorignore. Pre-PR, the describe was a step of the synthetic and got filtered out byonly; post-PR its tests will run alongside theonlytest.
If that's an accepted behavior change, please add a regression test that pins it down. If it's a regression, the fix is probably to propagate only/ignore from syntheticParent into the promoted child at registerAsDenoTest() time.
| const parent = this.syntheticParent; | ||
| if (parent) { | ||
| const { beforeAll } = parent.describe; | ||
| if (typeof beforeAll === "function") { | ||
| await beforeAll.call(context); | ||
| } else if (beforeAll) { | ||
| for (const hook of beforeAll) { | ||
| await hook.call(context); | ||
| } | ||
| } | ||
| try { | ||
| TestSuiteInternal.active.push(this.symbol); | ||
| await TestSuiteInternal.run(this, context, t); | ||
| } finally { | ||
| } | ||
| const { beforeAll } = this.describe; | ||
| if (typeof beforeAll === "function") { | ||
| await beforeAll.call(context); | ||
| } else if (beforeAll) { | ||
| for (const hook of beforeAll) { | ||
| await hook.call(context); | ||
| } | ||
| } | ||
| try { | ||
| if (parent) { | ||
| TestSuiteInternal.active.push(parent.symbol); | ||
| } | ||
| TestSuiteInternal.active.push(this.symbol); | ||
| await TestSuiteInternal.run(this, context, t); | ||
| } finally { | ||
| TestSuiteInternal.active.pop(); | ||
| if (parent) { | ||
| TestSuiteInternal.active.pop(); | ||
| const { afterAll } = this.describe; | ||
| } | ||
| const { afterAll } = this.describe; | ||
| if (typeof afterAll === "function") { | ||
| await afterAll.call(context); | ||
| } else if (afterAll) { | ||
| for (const hook of afterAll) { | ||
| await hook.call(context); | ||
| } |
There was a problem hiding this comment.
Minor: the typeof fn === "function" / for (const hook of …) pattern now appears four times in this one function (parent-before, this-before, this-after, parent-after). The existing run/runTest use the same pattern, so this is at least consistent — but a tiny runHooks(hooks, context) helper would tighten things up a lot. Optional.
| // When adding a top-level `it()` to the synthetic global suite, the global | ||
| // needs to become a real `Deno.test` so the test has a place to run. | ||
| // Child `describe`s are promoted at construction time and never reach | ||
| // `addStep` with the synthetic suite as their parent. | ||
| if ( | ||
| suite.isSynthetic && !suite.#registeredOptions && | ||
| !(step instanceof TestSuiteInternal) | ||
| ) { | ||
| suite.registerAsDenoTest(); |
There was a problem hiding this comment.
This is where the mixed case (top-level hook + top-level it() + nested describe) materializes: the synthetic gets registered here as a real Deno.test, while the sibling describes have already been promoted. End result is that the global beforeAll fires once per top-level Deno.test — twice or more across the file — rather than strictly once. The PR body calls this out as accepted, and I think that's fine, but please:
- Add a regression test that asserts the new behavior (two
Deno.testcalls, globalbeforeAllruns in each). Without one, a future refactor could silently change it back. - Consider a short JSDoc note near
addHookinbdd.ts(or in thetesting/bdd.tsmodule docs) so users mixing styles aren't surprised.
| name: "global", | ||
| [name]: fn, | ||
| }); | ||
| }, true); |
There was a problem hiding this comment.
Worth a one-line comment here explaining why true — e.g. // Mark as synthetic so it isn't eagerly registered as a Deno.test; see TestSuiteInternal#isSynthetic. The boolean literal at a call site is otherwise opaque.
Summary
A top-level
beforeAll/afterAll/beforeEach/afterEachcall creates a synthetic"global"TestSuiteto host the hook. Previously that suite was also eagerly registered as a top-levelDeno.test, so any nesteddescribewas reported as a child step ofglobal— inflating the step count by one and changing the test output.For example, this file:
reported
FAILED | 0 passed | 1 failed (3 steps)even though only two tests existed. With this PR it reports(2 steps), matching the file's contents.How the fix works
"global"suite is now only registered withDeno.testlazily — only if a top-levelit()is actually added to it. Otherwise the wrapper never surfaces as a counted test/step.describeof the synthetic suite is promoted to its own top-levelDeno.test. It keeps a back-reference to the synthetic suite (syntheticParent) so the global hooks still wrap its tests at run time:beforeAll/afterAllare invoked around the promoted child's body.symbolis pushed ontoTestSuiteInternal.activefor the duration of the run, sorunTest's normal traversal picks upbeforeEach/afterEachfrom the global suite just like any other parent.it()calls under a synthetic suite still work exactly as before: the synthetic suite is registered lazily on the firstit()and theit()s become its steps.Behavior preserved
describe→ fixed: nested describes register as their ownDeno.tests, hooks still run around their tests (regression test added).it()→ unchanged: a single"global"Deno.testwraps theit()s (existing tests still pass).it.only()propagation to the synthetic global → unchanged (existing test still passes).it()+ nesteddescribe) → describes are promoted; the synthetic wrapper remains for the top-levelit()s. Hooks fire around both, which meansbeforeAllruns once per top-levelDeno.testrather than strictly once across the file. Acceptance criteria only requires the hook still "runs around tests" in this case.Tests
Added three regression tests in
testing/bdd_test.ts:Deno.testnamed after the user'sdescribe, two steps, hooks fire correct number of times.Deno.test.Closes bartlomieju/orchid-inbox#68
Test plan
(2 steps)instead of(3 steps).deno test -A --parallel --no-check testing/— 137 passed (159 steps), 0 failed.deno test -A --parallel --no-check expect/— 167 passed, 0 failed (expectbuilds onbdd).deno fmt --check— clean.deno lint— clean.deno test --doc testing/bdd.ts— doc examples still pass.